Printed circuit board (PCB) virtual x-ray visualization

ABSTRACT

A method for visualizing a reference object in an augmented reality (AR) workspace includes: calibrating the AR workspace by mapping a first coordinate space to the AR workspace; projecting an interface with control elements onto the AR workspace; obtaining an image of the AR workspace that includes the interface and the reference object; identifying the reference object in the image using an image recognition algorithm; retrieving a reference file associated with the reference object, where the reference file includes structural information describing one or more layers of the reference object; identifying a first point of interest (POI) on the reference object; generating, based on the structural information of the reference file, a virtual x-ray representation of structures of the reference object; projecting the virtual x-ray representation onto the AR workspace. The virtual x-ray representation includes a rendering of an internal structure of the reference object.

BACKGROUND

Augmented Reality (AR) allows a user to view and/or interact with acomputer-generated output overlaid on or around a physical object in thereal-world environment. In some cases, the computer-generated output mayinclude information that cannot be perceived by the user that isobserving or interacting with the physical object. For example, aprinted circuit board (PCB) consists of layers of an insulatingepoxy-composite material sandwiched together with layers of conductivetraces, pads, and/or electronic components. When diagnosing or repairingPCBs, it can be useful for the user to be able to see the full structureof a board, including a schematic of an internal layer of the PCB, whichmay not be visible while looking directly at the physical object.

SUMMARY

In general, one or more embodiments of the invention relate to a methodfor visualizing a reference object in an augmented reality (AR)workspace. The method comprises: calibrating the AR workspace by mappinga first coordinate space to the AR workspace; projecting an interfacewith control elements onto the AR workspace; obtaining an image of theAR workspace that includes the interface and the reference object;identifying the reference object in the image using an image recognitionalgorithm; retrieving a reference file associated with the referenceobject, where the reference file includes structural informationdescribing one or more layers of the reference object; identifying afirst point of interest (POI) on the reference object; generating, basedon the structural information of the reference file, a virtual x-rayrepresentation of structures of the reference object located at thefirst POI; projecting the virtual x-ray representation onto the ARworkspace. The virtual x-ray representation includes a rendering of aninternal structure of the reference object at the first POI.

In general, one or more embodiments of the invention relates to anon-transitory computer readable medium (CRM) storing computer readableprogram code for visualizing a reference object in an augmented reality(AR) workspace. The computer readable program code causes a computer to:calibrate the AR workspace by mapping a first coordinate space to the ARworkspace; project an interface with control elements onto the ARworkspace; obtain an image of the AR workspace that includes theinterface and the reference object; identify the reference object in theimage using an image recognition algorithm; retrieve a reference fileassociated with the reference object, where the reference file includesstructural information describing one or more layers of the referenceobject; identify a first point of interest (POI) on the referenceobject; generate, based on the structural information of the referencefile, a virtual x-ray representation of structures of the referenceobject located at the first POI; project the virtual x-rayrepresentation onto the AR workspace. The virtual x-ray representationincludes a rendering of an internal structure of the reference object atthe first POI.

In general, one or more embodiments of the invention relates to a systemfor visualizing a reference object in an augmented reality (AR)workspace. The system comprises: a memory; and a processor coupled tothe memory. The processor is configured to: calibrate the AR workspaceby mapping a first coordinate space to the AR workspace; project aninterface with control elements onto the AR workspace; obtain an imageof the AR workspace that includes the interface and the referenceobject; identify the reference object in the image using an imagerecognition algorithm; retrieve a reference file associated with thereference object from the memory, where the reference file includesstructural information describing one or more layers of the referenceobject; identify a first point of interest (POI) on the referenceobject; generate, based on the structural information of the referencefile, a virtual x-ray representation of structures of the referenceobject located at the first POI; project the virtual x-rayrepresentation onto the AR workspace. The virtual x-ray representationincludes a rendering of an internal structure of the reference object atthe first POI.

Other aspects of the invention will be apparent from the followingdescription and the appended claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a system in accordance with one or more embodiments of theinvention.

FIG. 2 shows a flowchart in accordance with one or more embodiments ofthe invention.

FIG. 3 shows a flowchart in accordance with one or more embodiments ofthe invention.

FIG. 4 shows a flowchart in accordance with one or more embodiments ofthe invention.

FIGS. 5A-5H show implementation examples in accordance with one or moreembodiments of the invention.

FIG. 6 shows a computing system in accordance with one or moreembodiments of the invention.

DETAILED DESCRIPTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures. Like elements in the variousfigures are denoted by like reference numerals for consistency.

In the following detailed description of embodiments of the invention,numerous specific details are set forth in order to provide a morethorough understanding of the invention. However, it will be apparent toone of ordinary skill in the art that the invention may be practicedwithout these specific details. In other instances, well-known featureshave not been described in detail to avoid unnecessarily complicatingthe description.

Throughout the application, ordinal numbers (e.g., first, second, third)may be used as an adjective for an element (i.e., any noun in theapplication). The use of ordinal numbers is not to imply or create aparticular ordering of the elements nor to limit any element to beingonly a single element unless expressly disclosed, such as by the use ofthe terms “before,” “after,” “single,” and other such terminology.Rather the use of ordinal numbers is to distinguish between theelements. By way of an example, a first element is distinct from asecond element, and the first element may encompass more than oneelement and succeed (or precede) the second element in an ordering ofelements.

Interacting with objects in a real-world environment is often limited bya user's physical senses (e.g., touch and sight). For example, whenworking on a multi-layer PCB, the user is limited to observingelectronic components (e.g., traces, pads, through holes, active/passivecircuit elements) that are disposed on the surface of the PCB facing theuser. PCBs are typically designed using software (e.g., computer-aideddesign (CAD) software) that allows for viewing multiple layerssimultaneously. However, relating a design schematic from the screen ofa computer to the physical PCB on a workbench board can be difficult,especially for PCBs with multiple layers, complex/dense/miniaturizedlayout designs, and/or visually similar pattern regions. Therefore, amethod of relating the design schematic, including inaccessible internalregions at a given point on a PCB, to the physical workspace around thePCB may be useful to a user. It will be appreciated that the presentinvention is not limited to visualizing internal regions of PCBs andthat the present invention may be applied to any reference object thatincludes an internal structure.

In general, embodiments of the invention provide a method, anon-transitory computer readable medium (CRM), and a system forvisualizing a reference object in an augmented reality (AR) workspace.More specifically, embodiments of the invention are directed toprocessing an image of the reference object and projecting a virtualx-ray representation of the reference object into the AR workspace toaid a user's understanding of the internal structural arrangement of thereference object. The virtual x-ray representation may include one ormore views of internal structures based on information retrieved from areference file (e.g., a collection of one or more computer aided design(CAD) files, electronic schematic files, diagnostic files, and/or anyother appropriate files that include information about the referenceobject). Furthermore, the virtual x-ray representation may includeadditional information blocks or visual indicator blocks to help theuser correlate the internal structures indicated by the virtual x-rayrepresentation with the physical reference object in the AR workspace.

The AR workspace may be a surface of any type (e.g., a desk, a wall, awhiteboard, a PCB platen) that is configured to display a projected ARoverlay. The AR workspace may comprise one or more mats with acalibrated work surface, ordinary work surfaces (e.g., a desk),irregular work surfaces (e.g., a textured or uneven surface), spatiallyseparated surfaces, any combination thereof, but is not particularlylimited to these configurations.

FIG. 1 shows a system (100) in accordance with one or more embodimentsof the invention. The system (100) has multiple components, and mayinclude, for example, a buffer (102), an imaging engine (108), a parsingengine (110), a transformation engine (112), a rendering engine (114),and a display engine (116). Each of these components (102, 108, 110,112, 114, and 116) may be located on the same computing device (e.g.,Projection with Interactive Capture (PIC) device, personal computer(PC), laptop, tablet PC, smartphone, multifunction peripheral, kiosk,server) or on a combination of different computing devices connected bya network of any size having wired and/or wireless connections. Each ofthe components of the system (100) is described in further detail below.

The buffer (102) may be implemented in hardware (i.e., circuitry),software, or any combination thereof. The buffer (102) is configured tostore an AR workspace image (104), a virtual x-ray representation (106),and a transformation set (H). Multiple AR workspace images (104),virtual x-ray representations (106), or transformation sets (H) may bestored in the buffer (102).

An AR workspace image (104) is an image of an AR workspace. Accordingly,the AR workspace image (104) may be a single image or a composite ofmultiple images to accommodate the geometry of the AR workspace. The ARworkspace image (104) may be saved in the buffer (102) in any imagingformat (e.g., bitmap file format, JPEG image, GIF image, TIFF image, PDFdocument).

In one or more embodiments, the AR workspace image (104) captures aninterface region, a reference region, and a projection region of the ARworkspace, each of which is described below. The interface region,reference region, and projection region may be spatially distinct fromeach other, partially overlap, or completely overlap. Although the ARworkspace is described as having three regions, in other embodiments ofthe invention, the AR workspace may have more or fewer regions.Furthermore, the functionality of each region described above may beshared among multiple regions or performed by a different region.

The interface region of the AR workspace is a region of any size orshape that includes an interface of the AR workspace. The interface mayinclude one or more control elements (e.g., virtual buttons, knobs,sliders, etc.) for user interaction. For example, the interface may bepart of an AR overlay that is projected onto the surface of the ARworkspace. By capturing the interface in the AR workspace image (104),the user may activate controls within the AR workspace without divertingattention away from the reference object in the AR workspace.Furthermore, the interface in the AR workspace may be supplemented byphysical controls and/or input devices of a connected computer device.

The reference region of the AR workspace is a region of any size orshape in the AR workspace that includes the reference object. Asdiscussed above, the reference object may be a PCB, multi-layer PCB, orany physical object with an internal structure. By capturing thereference region in the AR workspace image (104), the system (100) mayactively track the location and orientation of the reference object toupdate the virtual x-ray representation (106).

The projection region is a region of any size or shape in the ARworkspace that includes the projected AR overlay, including the virtualx-ray representation (106). The projection region may be predefined andphysically separated from the reference region and/or the interfaceregion to simplify parsing of the AR workspace image (104). In one ormore embodiments, the reference region and projection region may overlapsuch that the projected AR overlay may include information projecteddirectly onto the reference object. By capturing the projection regionin the AR workspace image (104), the system (100) may monitor the user'sinteraction with the virtual x-ray representation (106) and contextuallyupdate the virtual x-ray representation (106).

The virtual x-ray representation (106) may include one or more views,magnified or at-scale, of the reference object. For example, a view thatis at the same scale as the reference object (i.e., not magnified) maybe most useful with a virtual x-ray representation (106) projected ontothe reference object. A view that is magnified may be most useful whenprojected into a separate projection region. In one or more embodimentswhere the reference object is a PCB, the virtual x-ray representation(106) may be a view that is produced by rendering the PCB design file(e.g., a CAD file) as a cropped and scaled image based on one or morepoints of interest on the PCB. As discussed in further detail below withrespect to the parsing engine (110) and rendering engine (114), thevirtual x-ray representation (106) may be updated based on the ARworkspace (e.g., movement of the reference object, movement of apointer, activation of controls). In general, the virtual x-rayrepresentation (106) may include a visual depiction of an internalstructure of the reference object that is projected into the real-worldenvironment.

The transformation set (H) may include one or more geometrictransformations (e.g., mathematical relationships) that define arelationship between two different coordinate systems (i.e., coordinatespaces). For example, when a projector projects an image onto an ARworkspace and a camera captures an image of the AR workspace, theprojected image and a captured image of the projection may not beidentical due to differences between the projector and camera (e.g.,spatial separation, resolution, field of view, color space, etc.).Geometric transformations may be used to reconcile one or more of thedifferences (e.g., scale, rotation, perspective, resolution, etc.)between the projected image and the captured image of the projection. Byapplying an appropriate transformation to the captured image, thetransformed captured image may appear identical to the projected image.In one or more embodiments, the transformation may be reversible (i.e.,when an inverse of the transformation is applied to the projected image,the transformed projected image may appear identical to the capturedimage).

In one or more embodiments, the transformation set (H) includes atransformation between a first coordinate space of the AR workspace(e.g., the projection region) and a second coordinate space of thereference object (e.g., the reference region). Therefore, thetransformation set (H) may include transformations such as: acamera-to-projector transformation between the camera coordinate spaceand the projector coordinate space; a projector-to-world transformationbetween the projector coordinate space and the first coordinate space ofthe AR workspace and/or the second coordinate space of the referenceobject; a camera-to-world transformations between the camera coordinatespace and the first coordinate space of the AR workspace and/or thesecond coordinate space of the reference object. Furthermore, anyappropriate single transformation (e.g., rotation, translation, scale,skew, cropping, or any appropriate image processing function) orcombination of transformations, such as a convolution of one or moretransformations may be included in the transformation set (H). By usingthe transformation set (H), virtual and real locations in the ARworkspace may be spatially related with greater accuracy.

The system (100) includes an imaging engine (108). The imaging engine(108) may be implemented in hardware (i.e., circuitry), software, or anycombination thereof. The imaging engine (108) obtains the AR workspaceimage (104) with a 2-dimensional and/or 3-dimensional imager (e.g., oneor more cameras that operate any appropriate wavelength range, a depthcamera, a hyperspectral camera) that records digital images of the ARworkspace. In one or more embodiments, the imaging engine (108) includesa camera of a Projection with Interactive Capture (PIC) device that alsoincludes a projector.

In one or more embodiments, a user may set a timing such that theimaging engine (108) automatically obtains an AR workspace image (104)at periodic intervals (e.g., continuous operation). Alternatively, theuser may asynchronously trigger the imaging engine (108) (e.g., byphysical control, virtual control, or gesture recognized by the system(100)) to obtain an AR workspace image (104). However, the methods tocontrol the timing and frequency of obtaining images are not limited tothose described herein and any appropriate timing may be used.

The system (100) further includes a parsing engine (110). The parsingengine (110) may be implemented in hardware (i.e., circuitry), software,or any combination thereof. The parsing engine (110) parses the ARworkspace image (104) to identify and extract information from the ARworkspace and associated reference files. Further, the parsing engine(110) may segment the AR workspace image (104) into multiple images thatmay be parsed individually (e.g., an image corresponding to thereference region, an image corresponding more specifically to thereference object, an image corresponding to the interface, an imagecorresponding to the projection region). The parsing engine (110) maymanipulate the AR workspace image (104) to improve the quality of theextracted content. For example, the parsing engine (110) may: apply atransform (e.g., rotation, scaling, and skew) to the AR workspace image(104); crop the AR workspace image (104); combine multiple AR workspaceimages (104). However, the methods of manipulating and parsing an ARworkspace image (104) are not limited to those described herein and thatany appropriate method may be used.

The parsing engine (110) may utilize the transformation set (H) toattribute a location information (i.e., coordinates in one or morecoordinate spaces) to objects identified in the AR workspace image(104). For example, the parsing engine (110) may calibrate the ARworkspace by mapping a first coordinate space to the AR workspace and asecond coordinate space to the reference object. In addition, theparsing engine (110) may generate or update one or more transformationsin the transformation set (H) based on objects identified in the ARworkspace image (104).

The parsing engine (110) may perform an optical character recognition(OCR) or image recognition (e.g., pattern, shape, and/or non-textcontent analysis) to identify, format, and extract content (e.g., areference object, a pointer, a text character, a non-text character, astring of text, a grid, a line, a drawing, a picture, a chart, an image,a graph, or a table) within an AR workspace image (104). Furthermore,the parsing engine (110) may determine a reference file associated withthe reference object (e.g., a computer aided design file, a diagnosticresults file, a manufacturing report, an invoice, etc.) based onidentifying information extracted from the AR workspace image (104). Thereference file may include multiple files (e.g., multipleversions/revisions of a CAD file) including files with multiple types ofinformation (e.g., structural, diagnostic, inventory, engineering,marketing, logistic information). The reference file may be stored andaccessed from the buffer (102) or from the memory of a networkedcomputing device (e.g., personal computer, smart phone, server, cloudserver).

The parsing engine (110) may use an image recognition algorithm toidentify and locate a pointer within an AR workspace image (104). Theimage recognition algorithm may be the same as or different from theimage recognition algorithm used to identify the reference object. Thepointer may be a hand, a finger, a stylus, a pen/pencil, a solderingiron, a knife, a scribe, or any appropriate tool for indicating a pointof interest in the AR workspace. In one or more embodiments, the parsingengine (110) may identify a point of interest at a predetermined offsetfrom the pointer (e.g., a predetermined distance from a fingertip orstylus to improve visibility for the user). The parsing engine (110) mayutilize the transformation set (H) to determine a location of thepointer or a point of interest in one or more coordinate spaces. Forexample, the parsing engine (110) may identify a first pointer locatedon the reference object and a second pointer located in a region of thevirtual x-ray representation.

Any common or proprietary parsing program (e.g., a word recognitionprogram, a table recognition program, an image recognition program) thatis able to identify and extract information from an image or a referencefile may be utilized by the parsing engine (110). The parsing engine(110) may store the extracted content in the buffer (102) to generate orupdate the virtual x-ray representation (106).

The system (100) includes a transformation engine (112). Thetransformation engine (112) may be implemented in hardware (i.e.,circuitry), software, or any combination thereof. The transformationengine (112) computes and applies transformations of the transformationset (H) to images captured by the imaging engine (108), images renderedby the rendering engine (114), and images projected by the displayengine (116).

Any program (e.g., an image processing program) that is able tomanipulate images may be utilized by the transformation engine (112).Furthermore, the transformation engine (112) may work in combinationwith one or more other engines of the system (100) to generate, update,and/or apply transformations to an image (e.g., generate and store atransformation that relates a first coordinate space of the AR workspaceand the second coordinate space). For example, the transformation engine(112) may apply a cropping transformation, to prepare an image for theparsing engine (110), based on information about the size and shape ofthe various regions of the AR workspace. The transformation engine (112)may map a first coordinate space to the entire AR workspace based on aregistration feature on a surface of the AR workspace that is identifiedby the parsing engine (110). In one or more embodiments where thereference object is a PCB, the transformation engine (112) may map asecond coordinate space to the PCB based on a registration feature ofthe PCB that is identified by the parsing engine (110).

The system (100) includes a rendering engine (114). The rendering engine(114) may be implemented in hardware (i.e., circuitry), software, or anycombination thereof. The rendering engine (114) renders AR overlayimages comprising the virtual x-ray representation (106) that are to beprojected by the display engine (116) into the AR workspace. Anyrendering software (e.g., a program for viewing and/or editing a wordfile, an image file, a CAD file, etc.) that is able to render an imagemay be utilized by the rendering engine (114). For example, therendering engine (114) may access a reference file to obtain structuralinformation for rendering structures in one or more layers of thereference object. In addition, the rendering engine (114) may utilizethe transformation set (H) to render images with an appropriate scaleand position to align physical and virtual objects in the AR workspace.A rendering may consist of one or more predefined views (e.g., a planview, an orthographic view, an isometric view, one or more views from apredetermined direct).

In one or more embodiments where the reference is a PCB, the renderingengine (114) may generate a plan view of the PCB for each discrete layeror surface of the PCB, as described in one or more reference files, andstore the images as virtual layers of the virtual x-ray representation(106). Further, the rendering engine (114) may combine one or morerendered images to illustrate a relationship between structures of thePCB (e.g., a conductive via connecting adjacent layers, a conductivetrace that connects electrical components within a layer). The renderingengine (114) may control the size, color palette, orientation,magnification, or any combination thereof for each virtual layers of thevirtual x-ray representation (106) to distinguish between differentstructures/layers of the PCB.

In one or more embodiments, the rendering engine (114) may furtherrender an information block describing the PCB or a point of interest onthe PCB. For example, the rendering engine (114) may render or highlighta point of interest on the PCB that includes an abnormal structureidentified by a diagnostic information in the reference file. Further,the rendering engine (114) may render an indicator block that correlatestwo locations in the AR workspace. For example, the indicator block mayinclude a connecting line between a feature rendered in the virtualx-ray representation (106) and the corresponding feature on the physicalreference object. Alternatively, or in addition, the indicator block mayvisually correlate the information block with a feature on the referenceobject and/or in the virtual x-ray representation (106).

The rendering engine (114) may include one or more controls (virtual orphysical) for a user to manipulate the contents of the AR overlay or thevirtual x-ray representation (106). For example, the control elements ofthe interface projected into the AR workspace may be used to filter thetype of structures or layers of the reference file that are rendered inthe virtual x-ray representation (106). In one or more embodiments, thecontrols may control a size, color palette, orientation, magnification,or any combination thereof for the virtual x-ray representation (106).However, the controls of the rendering engine (114) are not limited tothe examples described herein and any appropriate controls may beincluded to provide the desired AR overlay.

The system (100) includes a display engine (116). The display engine(116) may be implemented in hardware (i.e., circuitry), software, or anycombination thereof. The display engine (116) may project the interfaceand the virtual x-ray representation (106) onto the AR workspace usingone or more lights, lasers, effects, and/or projections. In addition,the display engine (116) may additionally display a version of virtualx-ray representation (106) on an output device of the system (100) or aconnected computing device. In one or more embodiments, the displayengine (116) includes a projector of a PIC device.

The display engine (116) may include a 2-dimensional and/or3-dimensional projector (e.g., a projector or a combination of one ormore projectors) that is able to project a digital image onto the ARworkspace. The display engine (116) may further include a 2-dimensionaldisplay (e.g., a liquid crystal display (LCD), a light emitting diode(LED) display, a cathode ray tube (CRT) display, or a thin filmtransistor (TFT) display), a 3-dimensional display, or a combination ofone or more displays that is able to display an AR overlay. The displayengine (116) may operate in an unrestricted color space in visible ornon-visible wavelength regimes (e.g., ultraviolet, visible, nearinfrared, and infrared).

The display engine (116) may project or display an interface, aninformation block, and/or an indicator block onto the AR workspace tofacilitate user interaction with the AR workspace. In a non-limitingexample, the interface may include one or more control elements thatallow the user to control the size, position, orientation, and contentof the virtual x-ray representation (106). In other words, the user maynavigate and manipulate the AR workspace using virtual controls withouthaving to reference a separate computing device.

In response to the rendering engine (114) rendering a virtual x-rayrepresentation (106), the display engine (116) may project the virtualx-ray representation (106) as part of an AR overlay in the AR workspace.The virtual x-ray representation (106) may be projected within apredetermined region of the AR workspace. The size, shape, orientation,and position of the virtual x-ray representation (106) may be determinedby the display engine (116) or the rendering engine (114).

Although the system (100) is shown as having six components (102, 108,110, 112, 114, and 116), in other embodiments of the invention, thesystem (100) may have more or fewer components. Furthermore, thefunctionality of each component described above may be shared amongmultiple components or performed by a different component. In addition,each component (102, 108, 110, 112, 114, and 116) may be utilizedmultiple times in serial or parallel to carry out a repeated operationor an iterative operation.

FIG. 2 shows a flowchart for visualizing a reference object in an ARworkspace in accordance with one or more embodiments of the invention.In a non-limiting example, the user may place a PCB in the AR workspaceand wish to visualize an internal structure of the PCB. Thus, the system(100) will obtain an AR workspace image (104) and subsequently generateand project a virtual x-ray representation (106) that includes a visualdepiction of the internal structure of the reference object.

At S210, the display engine (116) projects an interface with controlelements onto an AR workspace. By including the interface in the ARworkspace, parsing of the AR workspace image (104) allows the user toexert control over the AR workspace without diverting attention awayfrom the AR workspace. To project the interface into the interfaceregion of the AR workspace, the system (100) may calibrate the ARworkspace by mapping a first coordinate space to the AR workspace. Asdiscussed above, the interface may be generated by the rendering engine(114) and projected by the display engine (116).

At S220, the imaging engine (108) obtains an AR workspace image (104)that includes the interface and the reference object to be visualized(e.g., a multi-layer PCB).

At S225, the system (100) determines whether the reference object in theAR workspace image (104) has been identified by the parsing engine(110). As discussed above, the parsing engine (110) parses the ARworkspace image (104) and may automatically identify and extractinformation from the image. The process of identifying the referenceobject will be discussed in further detail below with respect to FIG. 3.

When the determination at S225 is NO (i.e., the reference object is notidentified by the parsing engine (110)), the process continues withS230.

At S230, the system (100) optionally waits for user input or a change inthe AR workspace. For example, the user may manually identify thereference object (e.g., enter the information via a connected computingdevice) or move/reorient the reference object to provide a differentperspective of the reference object that the parsing engine (110) may beable to analyze from a subsequent AR workspace image (104). In one ormore embodiments, the imaging engine (108) may automatically obtain anew AR workspace image (104) at periodic intervals. Alternatively, theuser may asynchronously trigger the imaging engine (108) (e.g., byphysical control, virtual control, or gesture recognized by the parsingengine (110)) to obtain a new AR workspace image (104) when thereference object is repositioned. In one or more embodiments, the system(100) skips S230 and immediately returns to S220 to obtain a new ARworkspace image (104) for analysis.

When the determination at S225 is YES (i.e., the reference object issuccessfully identified by the parsing engine (110)), the processcontinues with S240. As describe below with respect to FIG. 3 ,identifying the reference object may include determining a referencefile associated with the reference object based on identifyinginformation provided by a user or extracted from the AR workspace image(104).

At S240, the parsing engine (110) retrieves the reference fileassociated with the reference object to obtain structural informationdescribing one or more structures of the reference object.

At S242, the rendering engine (114) optionally generates an informationblock describing the reference object and the display engine (116) mayproject the information block onto the AR workspace. The informationblock may be separate from or integrated as part of a virtual x-rayrepresentation (106). For example, the information block may present theuser with general information (e.g., date, time) or information aboutthe reference object based on the content of the reference file (e.g.,filename of the reference file, product name, product type, size,dimensions, etc.)

In one or more embodiments where the information block is a component ofthe virtual x-ray representation (106), the information block maypresent the user with specific information describing one or morestructures of the reference object. In other words, the virtual x-rayrepresentation (106) may be generated and stored before a specific pointof interest on the reference object is identified by the user of thesystem (100). For example, the virtual x-ray representation (106) mayinclude a two-dimensional or three-dimensional rendering of the entirereference object projected in the information block.

At S245, the system (100) determines whether a point of interest (POI)on the reference object (i.e., a first POI) has been identified in theAR workspace image (104) by the parsing engine (110). As discussedabove, the parsing engine (110) may identify a point of interest basedon a pointer (e.g., a hand, a finger, a stylus, a pen/pencil, asoldering iron, or any appropriate tool) that has been identify by animage or shape recognition algorithm. The process of identifying a pointof interest will be discussed in further detail below with respect toFIG. 4 .

When the determination at S245 is NO (i.e., a point of interest is notidentified by the parsing engine (110)), the process returns to S230.

When the determination at S245 is YES (i.e., a point of interest issuccessfully identified by the parsing engine (110)), the processcontinues with S250.

At S250, the display engine (116) projects a virtual x-rayrepresentation (106) of the reference object onto the AR workspace. Asdiscussed above, the virtual x-ray representation (106) may include ormay be independent of an information block. The virtual x-rayrepresentation (106) may include a visual depiction of one or moreinternal structures of the reference object located at or in thevicinity of the first POI identified by the parsing engine (110).

In addition, rather than stopping at the projection of the virtual x-rayrepresentation (106) onto the AR workspace, the system (100) may repeatthe above process from S220 to actively update the virtual x-rayrepresentation (106) as the user interacts with the AR workspace. Thesystem (100) may update the virtual x-ray representation (106) to trackthe first POI as a user moves a pointer around the reference object. Thesystem (100) may update the virtual x-ray representation (106) based onthe user switching a tool used as the pointer. The system (100) maychange the virtual x-ray representation (106) to show a new referenceobject introduced into the AR workspace or switch between multiplereference objects as the user interacts with them.

In one or more embodiments where the reference object is a multi-layerPCB, S250 may include generating and projecting a virtual x-rayrepresentation (106) based on a CAD file comprising structuralinformation for the entire PCB. The virtual x-ray representation (106)may show a plan view, centered on the first POI, of every layer of thePCB. Each layer of the PCB may correspond to one or more virtual layersof the virtual x-ray representation (106). Each virtual layer of thevirtual x-ray representation (106) may be color-coded to differentiatebetween virtual layers and/or to demonstrate relationships betweendifferent virtual layers. The virtual x-ray representation (106) mayshow a plan view of a single layer of the PCB, internal or external,based on a selection input into the interface by the user.

In one or more embodiments, the system (100) may contextually update thevirtual x-ray representation (106) based on a tool that the user uses asthe pointer (e.g., a stylus, a scribe, a knife, a soldering iron). Forexample, in the case of a soldering iron, the rendering engine (114) maychange the selected structural layer of the reference file to a surfacelayer that is accessible to the soldering iron and/or may filter therendering to only show components that may be modified by a solderingiron. Furthermore, the rendering engine (114) may update an informationblock to include new or different information related to the newlyselected tool.

FIG. 3 shows a flowchart for identifying the reference object in the ARworkspace in accordance with one or more embodiments of the invention.

At S310, the parsing engine (110) identifies a registration feature onthe reference object using an image recognition algorithm. Theregistration feature may be any identifying information or visuallandmark on the reference object. In one or more embodiments where thereference object is a multi-layer PCB, the registration feature may beany identifying information disposed on the PCB (e.g., a label, abarcode, a symbol/logo) or any structural landmark on the PCB (e.g., anedge, an electronic component, a conductive trace, a through-hole).

At S320, the parsing engine (110) maps a second coordinate space to thereference object based on the registration feature. Because the secondcoordinate space is based on the registration feature of the referenceobject, the second coordinate space may be consistent with a coordinatespace used by a reference file. In one or more embodiments where thereference file is a CAD file, the second coordinate space may define alocal origin consistent with a conventional origin in a CAD processingprogram (e.g., at a corner or at the center of the reference object).

At S330, the parsing engine (110) generates a transformation thatrelates the first coordinate space and the second coordinate space. Byrelating the first and second coordinate spaces with a transformation ora transformation set (H), the system (100) may directly relate locationinformation in the AR workspace (e.g., a pointer location) to a relativelocation on the reference object. Subsequently, the relative location onthe reference object may be used to identify relevant information in areference file. For example, in the non-limiting example of a CADreference file, when the pointer in the first coordinate system overlapsa portion of the reference object, the transformation engine (112) maycalculate a point of interest in the second coordinate system (i.e., inthe coordinate system of the CAD file) based on the transformation.

At S340, the parsing engine (110) determines a reference file associatedwith the reference object. As discussed above, the parsing engine (110)may determine the reference file (e.g., a computer aided design file, adiagnostic results file, a manufacturing report, an invoice, etc.)associated with the reference object based on identifying informationextracted from the AR workspace image (104). Alternatively, the parsingengine (110) may identify the reference file based on informationprovided by the user.

In one or more embodiments where the reference object is a PCB, theparsing engine (110) may identify the PCB based on a registrationfeature, identifying information on the surface of the PCB (e.g.,printed text, an applied label, a computer readable code such as barcodeor QR code), a shape of the PCB, an arrangement of components disposedon the PCB, or any combination thereof. For example, the registrationfeature may be the identifying information on the surface of the PCB orany structural landmark on the PCB. Furthermore, the parsing engine(110) may determine a reference file associated with the PCB (e.g., acomputer aided design file, a diagnostic results file, a manufacturingreport, an invoice, etc.) based on registration feature or identifyinginformation extracted from the AR workspace image (104).

FIG. 4 shows a flowchart for identifying a point of interest in the ARworkspace, in accordance with one or more embodiments of the invention.In a non-limiting example, the user may use one or more pointers tointeract with the AR workspace. As discussed above, a pointer may be ahand, a finger, a stylus, a pen/pencil, a soldering iron, or anyappropriate tool for indicating a point of interest in the AR workspace.Based on the location of each pointer and its associated point ofinterest, the user may input a control command to the system (100)and/or the system (100) may update the virtual x-ray representation(106). In one or more embodiments, multiple branches of FIG. 4 may beperformed simultaneously or sequentially to generate/update a virtualx-ray representation (106).

At S410, the parsing engine (110) identifies one or more pointers in theAR workspace. The parsing engine (110) may use an image recognitionalgorithm to identify and locate each pointer within an AR workspaceimage (104). The parsing engine (110) may identify a point of interest(POI) associated with each pointer.

At S415, the parsing engine (110) determines a location of each pointerin one or more coordinate spaces based on the transformation set (H).The parsing engine (110) may categorize the location of the pointerbased on an interface region, a reference region, and a projectionregion of the AR workspace. Although the process is shown as havingthree distinct branches, in other embodiments of the invention, theprocess may have more or fewer branches. Furthermore, the processes ofeach branch described below may be shared among multiple branches orperformed by a different branch. As discussed above, each branch may beutilized multiple times in serial or parallel to carry out a repeatedoperation or an iterative operation.

When the parsing engine determines the location of the pointer to be inthe reference region (e.g., overlapping or in the vicinity of thereference object), the process continues with S420.

At S420, the parsing engine (110) determines a location of the POI inthe first coordinate space based on the location of the pointer. In oneor more embodiments, the parsing engine (110) may locate the POI at apredetermined offset from the pointer to improve visibility for theuser.

At S422, the parsing engine (110) determines a location of the POI inthe second coordinate space based on the transformation set (H). Becausethe second coordinate space is based on the geometry of the referenceobject, the location of the POI may further be correlated with acoordinate space used by the reference file.

At S424, the system (100) updates the virtual x-ray representation (106)based on the location of the POI. For example, the parsing engine (110)may retrieve, from the reference file, information regarding thestructures of the reference object based on the location of the POI inthe second coordinate space. The rendering engine (114) may update thevirtual x-ray representation (106) and display engine (116) may projectthe updated representation into the AR workspace. In one or moreembodiments, the updated virtual x-ray representation (106) may besynchronized with a pointer that the user moves around the referenceobject (e.g., when the user tracks an electrical trace along the surfaceof a PCB).

When the parsing engine determines the location of the pointer to be inthe interface region (e.g., overlapping or in the vicinity of theinterface), the process continues with S430.

At S430, the parsing engine (110) determines a selected control of theinterface based on the location of the pointer.

At S432, the system (100) updates the virtual x-ray representation (106)based on the selected control. For example, the selected control changesone or more layers of the reference file that are rendered for displayin the virtual x-ray representation (106). The selected control maychange a size, color palette, orientation, magnification, or anycombination thereof of one or more renderings in the virtual x-rayrepresentation (106). The selected control may freeze the virtual x-rayrepresentation (106) such that the user may freely move about the ARworkspace without disrupting the virtual x-ray representation (106).However, the controls are not limited to the examples described hereinand any appropriate control over the AR workspace may be used.

When the parsing engine determines the location of the pointer to be inthe projection region (e.g., overlapping or in the vicinity of thevirtual x-ray representation (106)), the process continues with S440.

At S440, the parsing engine (110) determines a corresponding location onthe reference object based on the location of the pointer in the ARworkspace and the transformation set (H). In other words, the user mayuse a pointer to indicate a location within the virtual x-rayrepresentation (106) which the system (100) may identify based on theknown content of the virtual x-ray representation (106). By using thetransformation set (H), the parsing engine (110) may relate a locationof the pointer in projection region (i.e., in the first coordinatespace) with the corresponding location of the reference object (i.e., inthe second coordinate space).

At S442, the parsing engine (110) identifies a second POI on thereference object based on the corresponding location. In one or moreembodiments, second POI is located within the first POI that is renderedin the original virtual x-ray representation (106).

At S424, the system (100) updates the virtual x-ray representation (106)based on the second POI. For example, the parsing engine (110) mayretrieve, from the reference file, information regarding the structuresof the reference object based on the location of the second POI in thesecond coordinate space. The rendering engine (114) may update thevirtual x-ray representation (106) and display engine (116) may projectthe updated representation into the AR workspace.

In one or more embodiments, the updated virtual x-ray representation(106) may include additional information regarding the second POI (e.g.,an information block describing a structure located at the second POI).For example, the additional information may include a connecting linebetween a feature rendered in the virtual x-ray representation (106) andthe corresponding feature on the physical reference object (e.g., anindicator block that correlates the position of the pointer in the ARworkspace with the second POI on the reference object). Alternatively,or in addition, the indicator block may visually correlate aninformation block with the reference object and/or the virtual x-rayrepresentation (106).

FIGS. 5A-5H show implementation examples of a virtual x-rayrepresentation (106) in accordance with one or more embodiments of theinvention.

FIG. 5A shows a system (100) that includes an AR workspace (500) and aProjection with Interactive Capture (PIC) device (501). The PIC device(501) includes a camera that obtains images of the AR workspace (500)and a projector that projects an AR overlay onto a surface of the ARworkspace (500). The imaging engine (108) includes the camera of the PICdevice (501). The display engine (116) includes the projector of the PICdevice (501). The display engine (116) may further include a display oroutput device of a connected computing device.

The PIC device (501) may further comprise a mat (not shown) thatfunctions as the surface of the AR workspace (500). The mat may have amatte surface to minimize reflections/glare and improve the quality(e.g., contrast and brightness) of an AR workspace image (104) obtainedby the PIC device (501). The mat may be plastic, vinyl, or anyappropriate material for a work surface. However, the PIC device (501)may not include a mat and may use any surface as the AR workspace (500).

The PIC device (501) projects an AR overlay including an interface (50)onto the AR workspace (500), as shown by the arrows emitted from the PICdevice (501). Furthermore, the PIC device (501) captures an AR workspaceimage (104) that may include: the AR workspace (500); an interface(505); a PCB (510) (i.e., the reference object); and one or morepointers (520) (e.g., the user's hands).

The interface (505) includes a plurality of control elements (e.g.,virtual buttons) that the use may interact with by moving a pointer (52)to the interface (505) and touching the surface of the AR workspace(500) at the location of the control element. For example, a virtualcontrol element may be activated by a pressure sensor embedded in thesurface of the AR workspace (500) or by a depth sensor included in theimaging engine (108) of the PIC device (501).

The PCB (510) may be a multi-layer PCB with a plurality of registrationfeatures (512) and structures (514) disposed on one or more externalsurfaces or internal layers. The camera of the PIC device (501) maycapture an AR workspace image (104) with sufficient resolution toidentify registration features (512) and structures (514) of the PCB(510) using image recognition techniques. In one or more embodiments,the registration features (512) may be through-holes disposed at cornersof the PCB (510) and the structures (514) may be surface mountedcomponents (e.g., conductive traces, passive electronic components,active electronic components, etc.) or embedded components (e.g.,conductive traces, interlayer vias, embedded components, etc.) of thePCB (510).

FIG. 5B shows an AR workspace (500) of FIG. 5A with an AR overlay (600).The AR overlay (600) includes an information block (605) and a virtualx-ray representation (610) based on the pointer (520) located on the PCB(510). For example, the user may use a finger (i.e., pointer (520)) toindicate structure (514 a) as a first point of interest (POI) in the ARworkspace (500). The system (100) generates the AR overlay (600) with aninformation block (605) that describes the PCB (510) and with a virtualx-ray representation (610) based on the first POI. The virtual x-rayrepresentation (610) may include a magnified view of a layer of the PCB(510) centered at the first POI. The magnified view may be generatedbased on a CAD file with the electronics layout of the PCB (510) (i.e.,a reference file). The virtual x-ray representation (610) includes arendering (614 a) with a magnified view of structure (514) and thesurrounding region of the PCB (510) included in the first POI.

FIG. 5C shows the AR workspace (500) of FIG. 5B with an updated virtualx-ray representation (610). The rendering engine (114) may render thevirtual x-ray representation (610) with one or more virtual layersdepicting a rendering (614 a) of structure (514 a) and a rendering (616a) of other related structures (e.g., conductive traces) on PCB (510).For example, the virtual x-ray representation (610) may further includea rendering (616 a) with a magnified view of conductive traces that areconnected to the structure (514 a). In one or more embodiments, theinformation block (605) may be updated to display information regardingthe conductive traces (e.g., critical traces that should not be damaged,traces with an abnormality detected during diagnostic testing, tracesthat need to be repaired, etc.).

In one or more embodiments, rendering (614 a) and rendering (616 a) maybe based on a single layer of the reference CAD file. For example, theoriginal virtual x-ray representation (610) of FIG. 5B may have beenfiltered to exclude renderings of conductive traces on the same physicallayer as the structure (514 a) to simplify the visual depiction of thePCB (510) for the user. The updated virtual x-ray representation (610)of FIG. 5C, without the filter, renders all structures on the selectedlayer of PCB (510). In other words, the content of the virtual x-rayrepresentation (610) may be adapted to include multiple virtual layers(i.e., filters) based on the needs of the user. The virtual layers ofthe virtual x-ray representation (610) may be distinguished by colorpalette, opacity, patterns, any combination thereof, or any appropriatevisual indicator.

FIG. 5D shows the AR workspace (500) of FIG. 5B with another type ofupdated virtual x-ray representation (610). In one or more embodiments,the updated virtual x-ray representation (610) includes a rendering (614a) of the structure (514) located on a first layer of the PCB (510) anda rendering (616 b) of conductive traces located on a second layer ofthe PCB (510).

FIG. 5E shows the AR workspace (500) of FIG. 5B with another type ofupdated virtual x-ray representation (610) that uses multiple virtuallayers of the virtual x-ray representation (610) to depict multiplelayers and multiple filtered subsets of structures on the PCB (510).

FIG. 5F shows the AR workspace (500) of FIG. 5B with another type ofupdated virtual x-ray representation (610). In one or more embodiments,the x-ray representation (610) may be used as an input to update thevirtual x-ray representation (610). As in FIG. 5B, a first POI on PCB(510) is indicated by a first pointer (520 a) to generate the originalAR overlay (600) including an information block (605) and a virtualx-ray representation (610) with rendering (614 a). The user may requiremore information about the structure (514 a) represented by therendering (614 a) and may use a second pointer (520 b) to indicate therendering (614 a) in the projection region of the AR workspace (500).The parsing engine (110) identifies and locates the second pointer (520a) and determines a corresponding location on the PCB (510) based on thelocation of the second pointer (520 a) in the first coordinate space ofthe AR workspace (500) and the transformation set (H) related to thesecond coordinate space of the PCB (510). Based on the second pointer(520 a) in the virtual x-ray representation (610), the parsing engine(110) may identify a second POI on the reference object based on thecorresponding location, where the second POI is located within the firstPOI. The virtual x-ray representation (610) may be updated to include anindicator block that correlates the location of the second pointer (520a) in the first coordinate space with the second POI located at thecorresponding location on the PCB (510).

FIG. 5G shows an AR workspace (500) of FIG. 5A with another AR overlay(600). The AR overlay (600) includes a virtual x-ray representation(610) based on the pointer (520) indicating an embedded structure (514b) on the PCB (510). The virtual x-ray representation (610) may includea magnified view of a layer of the PCB (510) generated based on the CADfile with the electronics layout of the PCB (510) (i.e., a referencefile). Further, the virtual x-ray representation (610) includes arendering (616 a) of embedded conductive traces located on an internallayer of the PCB (510).

FIG. 5H shows the AR workspace (500) of FIG. 5G with an updated virtualx-ray representation (610) that uses multiple virtual layers of thevirtual x-ray representation (610) to depict multiple layers ofinformation regarding the embedded structures (514 b) on the PCB (510).The user may require more diagnostic information about the embeddedstructures (514 b) represented by the rendering (616 a) and may commandthe system (100) to visually highlight any abnormal structures (e.g.,shorted conductive traces, defective components, etc.) in the regionindicated by pointer (520). Based on the location of the pointer (520)in the second coordinate space of the PCB (510), the parsing engine(110) may identify a diagnostic file/dataset (e.g., bed-of-nail testreport, visual inspection report, manufacturing statistics) associatedwith the PCB (510) and extract information identifying abnormalstructures near embedded structures (514 b). For example, the referencefile may include the CAD file with the electronics layout of the PCB(510) and the diagnostic report.

In one or more embodiments, the diagnostic report may include a list ofregions with statistically higher rates of failure. Accordingly, theupdated virtual x-ray representation (610) may be updated withrenderings (616 b) of the problematic regions, an information block (notshown) describing the failures, and/or the indicator block (620) toassociate the rendering (616 b) with highlighted failure regionsprojected onto the PCB (510).

In one or more embodiments, the diagnostic report may include identifiedfailure points for the specific PCB (510). The virtual x-rayrepresentation (610) may be updated to include an additional virtuallayer with a rendering (616 b) of the failure points and/or locations tomake modification to the PCB (510) to circumvent the failure points. Asdiscussed above, the system (100) may also provide additional contextualinformation based on objects recognized by the parsing engine (110). Forexample, if a soldering iron (not shown) is introduced into the ARworkspace (500) as a second pointer, parsing engine (110) may identifythe soldering iron and instruct the rendering engine (114) to update thevirtual x-ray representation (610) with a route to solder a bypass wirearound an abnormal conductive trace or defective component.Alternatively, if a scribe or blade (not shown) is introduced into theAR workspace (500) as a second pointer, parsing engine (110) mayidentify the cutting tool and instruct the rendering engine (114) updatethe virtual x-ray representation (610) with locations to cut an abnormalconductive trace. Further, the virtual x-ray representation (610) may beupdated to include an indicator block (620) to associate the rendering(616 b) with a highlighted region projected onto the PCB (510) toindicate a location of the corrective modifications.

The parsing engine (110) determines a corresponding location on the PCB(510) based on the location of the second pointer (520 a) in the firstcoordinate space of the AR workspace (500) and the transformation set(H) related to the second coordinate space of the PCB (510). Based onthe second pointer (520 a) in the virtual x-ray representation (610),the parsing engine (110) may identify a second POI on the referenceobject based on the corresponding location, where the second POI islocated within the first POI. The virtual x-ray representation (610) maybe updated to include an indicator block that correlates the location ofthe second pointer (520 a) in the first coordinate space with the secondPOI located at the corresponding location on the PCB (510).

Embodiments of the invention may be implemented on virtually any type ofcomputing system, regardless of the platform being used. For example,the computing system may be one or more mobile devices (e.g., laptopcomputer, smart phone, personal digital assistant, tablet computer, orother mobile device), desktop computers, servers, blades in a serverchassis, or any other type of computing device or devices that includesat least the minimum processing power, memory, and input and outputdevice(s) to perform one or more embodiments of the invention. Forexample, as shown in FIG. 6 the computing system (700) may include oneor more computer processor(s) (702), associated memory (704) (e.g.,random access memory (RAM), cache memory, flash memory), one or morestorage device(s) (706) (e.g., a hard disk, an optical drive such as acompact disk (CD) drive or digital versatile disk (DVD) drive, a flashmemory stick), and numerous other elements and functionalities. Thecomputer processor(s) (702) may be an integrated circuit for processinginstructions. For example, the computer processor(s) may be one or morecores, or micro-cores of a processor. The computing system (700) mayalso include one or more input device(s) (710), such as a PIC device,camera, imager, touchscreen, keyboard, mouse, microphone, touchpad,electronic pen, or any other type of input device. Further, thecomputing system (700) may include one or more output device(s) (708),such as a PIC device, a projector, a screen (e.g., a liquid crystaldisplay (LCD), a plasma display, touchscreen, cathode ray tube (CRT)monitor, or other display device), a printer, external storage, or anyother output device. One or more of the output device(s) may be the sameor different from the input device(s). The computing system (700) may beconnected to a network (712) (e.g., a local area network (LAN), a widearea network (WAN) such as the Internet, mobile network, or any othertype of network) via a network interface connection (not shown). Theinput and output device(s) may be locally or remotely (e.g., via thenetwork (712)) connected to the computer processor(s) (702), memory(704), and storage device(s) (706). Many different types of computingsystems exist, and the aforementioned input and output device(s) maytake other forms.

Software instructions in the form of computer readable program code toperform embodiments of the invention may be stored, in whole or in part,temporarily or permanently, on a non-transitory computer readable mediumsuch as a CD, DVD, storage device, a diskette, a tape, flash memory,physical memory, or any other computer readable storage medium.Specifically, the software instructions may correspond to computerreadable program code that when executed by a processor(s), isconfigured to perform embodiments of the invention.

Further, one or more elements of the aforementioned computing system(700) may be located at a remote location and be connected to the otherelements over a network (712). Further, one or more embodiments of theinvention may be implemented on a distributed system having a pluralityof nodes, where each portion of the invention may be located on adifferent node within the distributed system. In one embodiment of theinvention, the node corresponds to a distinct computing device.Alternatively, the node may correspond to a computer processor withassociated physical memory. The node may alternatively correspond to acomputer processor or micro-core of a computer processor with sharedmemory and/or resources.

Embodiments of the invention may have one or more of the followingadvantages: the ability to visualize internal structure of a referenceobject without destructive testing methods or expensive equipment (x-rayscanners); the ability to project an AR overlay that provides structuralinformation in the same tactile environment as a reference object ratherthan on a separate computer monitor; the ability to visualize diagnosticinformation and failure location on a faulty PCB in any environment(e.g., a work/repair station equipped with a PIC device or part of a PICdevice); the ability to manipulate the internal visualization of thereference object in a real-world environment, with or without a separatecomputing system; the ability to rapidly and contextually conveydiagnostic information to expedite visual inspection and/or repair ofPCB s.

Although the disclosure has been described with respect to a limitednumber of embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that various other embodiments may bedevised without departing from the scope of the present invention.Accordingly, the scope of the invention should be limited only by theattached claims.

What is claimed is:
 1. A method for visualizing a reference object in anaugmented reality (AR) workspace, the method comprising: calibrating theAR workspace by mapping a first coordinate space to the AR workspace;projecting an interface with control elements onto the AR workspace;obtaining an image of the AR workspace that includes the interface andthe reference object; identifying the reference object in the imageusing an image recognition algorithm; retrieving a reference fileassociated with the reference object, where the reference file includesstructural information describing one or more layers of the referenceobject; identifying a first point of interest (POI) on the referenceobject; generating, based on the structural information of the referencefile, a virtual x-ray representation of structures of the referenceobject located at the first POI; projecting the virtual x-rayrepresentation onto the AR workspace, wherein the virtual x-rayrepresentation includes a rendering of an internal structure of thereference object at the first POI.
 2. The method of claim 1, wherein theidentifying of the reference object includes: identifying a registrationfeature on the reference object using the image recognition algorithm;mapping a second coordinate space to the reference object based on theregistration feature; generating a transformation that relates the firstcoordinate space and the second coordinate space; and determining thereference file associated with the reference object based on identifyinginformation provided by a user or extracted from the image.
 3. Themethod of claim 2, wherein the identifying of the first POI includes:identifying a first pointer that is located on the reference object;determining a location of the first POI in the first coordinate spacebased on the location of the first pointer; and determining a locationof the first POI in the second coordinate space based on thetransformation; retrieving, from the reference file, informationregarding the structures of the reference object based on the locationof the first POI in the second coordinate space.
 4. The method of claim3, further comprising: identifying a second pointer that is located in aregion containing the virtual x-ray representation; determining, for thesecond pointer, a corresponding location on the reference object basedon a location of the second pointer in the first coordinate space andthe transformation; identifying a second POI on the reference objectbased on the corresponding location, where the second POI is locatedwithin the first POI; and updating the virtual x-ray representation withinformation regarding the second POI.
 5. The method of claim 4, whereinthe virtual x-ray representation is updated to include an informationblock describing structures associated with the second POI.
 6. Themethod of claim 4, wherein the virtual x-ray representation is updatedto include an indicator block that correlates the location of the secondpointer in the first coordinate space with the corresponding location onthe reference object.
 7. The method of claim 3, further comprising:identifying a second pointer in the AR workspace that is located in aregion of the interface; determining a selected control from the controlelements of the interface based on a location of the second pointer inthe first coordinate space; and updating the virtual x-rayrepresentation based on the selected control.
 8. The method of claim 7,wherein the reference object is a printed circuit board (PCB), theselected control determines which internal layer of the PCB to includein the virtual x-ray representation.
 9. The method of claim 1, whereinthe reference object is a printed circuit board (PCB), the referencefile includes diagnostic information that identifies an abnormalstructure on the PCB, the first POI includes the abnormal structure, andthe virtual x-ray representation further includes a highlighted regionthat overlaps the PCB in the AR workspace to indicate a location of theabnormal structure on the PCB.
 10. The method of claim 1, wherein theimage of the AR workspace is obtained with a Projection with InteractiveCapture (PIC) device that comprises a camera and a projector, and thevirtual x-ray representation is projected into the AR workspace by theprojector.
 11. A non-transitory computer readable medium (CRM) storingcomputer readable program code for visualizing a reference object in anaugmented reality (AR) workspace, the computer readable program codecauses a computer to: calibrate the AR workspace by mapping a firstcoordinate space to the AR workspace; project an interface with controlelements onto the AR workspace; obtain an image of the AR workspace thatincludes the interface and the reference object; identify the referenceobject in the image using an image recognition algorithm; retrieve areference file associated with the reference object, where the referencefile includes structural information describing one or more layers ofthe reference object; identify a first point of interest (POI) on thereference object; generate, based on the structural information of thereference file, a virtual x-ray representation of structures of thereference object located at the first POI; project the virtual x-rayrepresentation onto the AR workspace, wherein the virtual x-rayrepresentation includes a rendering of an internal structure of thereference object at the first POI.
 12. The non-transitory CRM of claim11, wherein to identify the reference object, the computer readableprogram code causes the computer to: identify a registration feature onthe reference object using the image recognition algorithm; and map asecond coordinate space to the reference object based on theregistration feature; generate a transformation that relates the firstcoordinate space and the second coordinate space; and determine thereference file associated with the reference object based on identifyinginformation provided by a user or extracted from the image.
 13. Thenon-transitory CRM of claim 12, wherein to identify the first POI, thecomputer readable program code causes the computer to: identify a firstpointer that is located on the reference object; determine a location ofthe first POI in the first coordinate space based on the location of thefirst pointer; and determine a location of the first POI in the secondcoordinate space based on the transformation; retrieve, from thereference file, information regarding the structures of the referenceobject based on the location of the first POI in the second coordinatespace.
 14. The non-transitory CRM of claim 13, wherein the computerreadable program code causes the computer to: identify a second pointerin the AR workspace that is located in a region containing the virtualx-ray representation; determine, for the second pointer, a correspondinglocation on the reference object based on a location of the secondpointer in the first coordinate space and the transformation; identify asecond POI on the reference object based on the corresponding location,where the second POI is located within the first POI; and update thevirtual x-ray representation with information regarding the second POI.15. The non-transitory CRM of claim 14, wherein the computer readableprogram code causes the computer to update the virtual x-rayrepresentation by including an information block describing structuresassociated with the second POI.
 16. The non-transitory CRM of claim 14,wherein the computer readable program code causes the computer to updatethe virtual x-ray representation by including an indicator block thatcorrelates the location of the second pointer in the first coordinatespace with the corresponding location on the reference object.
 17. Thenon-transitory CRM of claim 13, wherein the computer readable programcode causes the computer to: identify a second pointer in the ARworkspace that is located in a region of the interface; determine aselected control from the control elements of the interface based on alocation of the second pointer in the first coordinate space; and updatethe virtual x-ray representation based on the selected control.
 18. Thenon-transitory CRM of claim 11, wherein the reference object is aprinted circuit board (PCB), the reference file includes diagnosticinformation that identifies an abnormal structure on the PCB, the firstPOI includes the abnormal structure, and the computer readable programcode causes the computer to generate the virtual x-ray representationwith a highlighted region that overlaps the PCB in the AR workspace toindicate a location of the abnormal structure on the PCB.
 19. Thenon-transitory CRM of claim 11, wherein the computer readable programcode causes the computer to: obtain the image of the AR workspace with aProjection with Interactive Capture (PIC) device that comprises a cameraand a projector, and project the virtual x-ray representation into theAR workspace with the projector of the PIC device.
 20. A system forvisualizing a reference object in an augmented reality (AR) workspace,the system comprising: a memory; and a processor coupled to the memory,where the processor is configured to: calibrate the AR workspace bymapping a first coordinate space to the AR workspace; project aninterface with control elements onto the AR workspace; obtain an imageof the AR workspace that includes the interface and the referenceobject; identify the reference object in the image using an imagerecognition algorithm; retrieve a reference file associated with thereference object from the memory, where the reference file includesstructural information describing one or more layers of the referenceobject; identify a first point of interest (POI) on the referenceobject; generate, based on the structural information of the referencefile, a virtual x-ray representation of structures of the referenceobject located at the first POI; project the virtual x-rayrepresentation onto the AR workspace, wherein the virtual x-rayrepresentation includes a rendering of an internal structure of thereference object at the first POI.